1. Field of the Invention
The present invention discloses a method of computing precise disparity using a stereo matching method based on developed census transform with an adaptive support weight method in area based stereo matching.
2. Description of the Related Art
Stereo matching is a technique of acquiring 3-dimensional information from 2-dimensional images acquired at positions different from each other at the same time through two cameras, corresponding points corresponding to the same point are found in left and right images, and disparity information between the two corresponding points is computed, thereby acquiring a depth map that is 3-dimensional distance information.
It has been known that a stereo matching method based on census transformation that is one of stereo matching methods is resistant to an error occurring by difference of gain or bias between two cameras, and is a method easy to be embodied by hardware as compared to other stereo matching methods such as sum of absolute difference (SAD), sum of squared difference (SSD), and normalized correlation (NCC).
FIG. 1 illustrates a stereo matching method based on census transformation using a disparity search range with a regular window size and a regular size in left and right images, FIG. 2 illustrates a method of computing a hamming weight window in a set window, and FIG. 3 illustrates a method of generating a hamming bit window using left and right hamming weight windows.
Referring to FIG. 1, a window centered on a left image point A is set to find a corresponding point of a right image about the left image point A, and a point having the highest correlation in a window area set in the right image about a correlation in a window area set in the left image is found while moving in a window set by the same size from a point B which corresponds to the point A in the right image corresponding thereto to a point B′ which is a maximum disparity prediction range, to obtain a disparity value between the left and right image corresponding to the point A.
As another example, in order to find a corresponding point in the right image about a point C, a point having the highest correlation in a regular displacement range from a point D to a point D′ is found.
A method for comparison of correlation between the left and right windows will be described with reference to FIG. 2 and FIG. 3.
The census transform is a transformation of generating bit stream information using brightness information of an image in a specific area.
FIG. 2 illustrates a method of computing a hamming weight when a census transform window is set to a 5×5 size, and a hamming weight has to be acquired for each census transform window of the left image and the right image.
A numeral displayed on each cell in a left census transform window in FIG. 2 represents a brightness value when an image is binarized, a size of each pixel is compared on the basis of center data, a value is set to 1 when a brightness value of each pixel is larger than a brightness value of the center point and otherwise is set to 0 to generate a hamming weight window as shown on the right, and a hamming bit stream as shown at a lower part of FIG. 2 is generated.
When each hamming weight window is generated about the left image and the right image, values of corresponding cells between a left image hamming weight window and a right image hamming weight window are compared with each other as shown in FIG. 3, and a hamming bit is set to 1 when two values are equal to each other and is set to 0 when two values are different from each other.
A lower part of FIG. 3 illustrates a hamming bit window, and illustrates that two images further coincides with each other as the number of hamming bits of 1 is increased.
The hamming weight value of a specific point of the left image is compared with the hamming weight values about the window set in the right image to extract a point with the largest number of hamming bits, and a disparity between the points of the left image and the right image corresponding thereto is computed.
Since a stereo matching method based on census transform is to compute a cost for similarity determination according to a window size, it has a characteristic that a matching precision is higher as a window size is larger. Accordingly, there are problems that an amount of computation for finding corresponding points is more rapidly increased as the window size becomes larger, and an amount of computation is a considerable time is consumed to acquire 3-dimensional information.
Meanwhile, an adaptive support weight method by which it is possible to obtain an effect of using windows having various sizes and shapes by putting a difference in each window area has been known.
The adaptive support weight method is a method of computing a weight using a distance from a center pixel and a color difference with respect to each pixel in windows set in a left image and a right image, and then finding a point having the highest correlation in the left and right image windows using an accumulated value of matching costs to which the weight of the left image and the weight of the right image are reflected, and a matching cost (dissimilarity) between a point p in the left image and a point q in the right image is represented as the following equation 1.
                                          D            ASW                    ⁡                      (                          p              ,                              p                d                                      )                          =                                            ∑                                                q                  ∈                                      N                    p                                                  ,                                                      q                    d                                    ∈                                      N                                          p                      d                                                                                            ⁢                                          w                ⁡                                  (                                      p                    ,                    q                                    )                                            ⁢                              w                ⁡                                  (                                                            p                      d                                        ,                                          q                      d                                                        )                                            ⁢                              e                ⁡                                  (                                      q                    ,                                          q                      d                                                        )                                                                                        ∑                                                q                  ∈                                      N                    p                                                  ,                                                      q                    d                                    ∈                                      N                                          p                      d                                                                                            ⁢                                          w                ⁡                                  (                                      p                    ,                    q                                    )                                            ⁢                              w                ⁡                                  (                                                            p                      d                                        ,                                          q                      d                                                        )                                                                                        [                  Equation          ⁢                                          ⁢          1                ]            
In the equation 1, a pixel p represents a center pixel of a window, q represents a peripheral pixel in, and Np and Np represents corresponding pixels in a left image window and a right image window area, respectively.
e(q, qd) represents a raw matching cost, and represents an absolute difference in a brightness value between pixels (p, q) of the corresponding left and right images.
      w    ⁡          (              p        ,        q            )        =      exp    ⁡          (              -                  (                                                    Δ                ⁢                                                                  ⁢                                  c                  pq                                                            γ                c                                      +                                          Δ                ⁢                                                                  ⁢                                  p                  pq                                                            γ                p                                              )                    )      represents a weight value, γc and γp are constant coefficients, Δcpq represents a color similarity between a pixel p and a pixel q, and Δppq represents a distance proximity between a pixel p and a pixel q.
However, the adaptive support weight method has a problem that an amount of computation is rapidly increased and a computation speed is increased, and has a problem that matching cost accuracy is lowered as compared with the census transform method since a brightness absolute difference between pixels is used as a raw matching cost.
In addition, there are a method of using only a specific area in a window designating a specific area for generating a bit stream of census transform using a sparse window shape, and a method of using only a specific area in a window area for computation in the adaptive support weight method. These methods have an advantage of decreasing an amount of computation and improving a computation speed, but have a problem of low accuracy as much. FIG. 4 illustrates an example of various window shapes, and various shapes are used besides the exemplified shapes.